1. Field of the Invention
The Present invention relates to the field of computer graphics, and in particular to Nonlinear Image Mapping (NLIM).
2. Description of Related Art
Since the earliest days of 3D computer graphics, generated images have been displayed on non-planar screens. Shapes such as spherical domes, toroidal sections, polyhedrons, and other types of surfaces have been used as screens. A variety of projector technologies have been used over the years to illuminate these screens, ranging from raster and calligraphic CRTs to light valves, LCDs, LCOS, DLP, and even scanning laser projectors. With the exception of laser projectors, all of these display devices direct their light through one or more lenses.
Unlike a pinhole camera, which projects light without any defocusing or distortion, lenses are configured for a specific optical geometry, and any deviation from that geometry can introduce geometric distortion. Lenses themselves may introduce photometric distortion, as is the case with a fisheye lens. In the past, custom optics such as light valve-driven domes have been crafted to enable images to be projected onto curved display surfaces without distortion. Creating such optics is very costly, as they are limited to a single geometry that can conflict with the purpose of the display e.g., locating the lens at the center of the dome.
Even with such custom optics, the raster pattern of the projection device may need to be adjusted in order for a raster image to be properly displayed. In a three-tube CRT projector, for example, the electron beam can be made to trace curves instead of straight lines using such mechanisms as the “pincushion” adjustment. By adjusting the sampling pattern inside the projector such that the raster grid is perceived to be regular (non-distorted), a three-dimensional scene may be rendered using conventional linear perspective projection without distortion artifacts.
Today, most projector optics are designed to project onto flat screens, with at most “keystone” distortion, which can be corrected with a simple linear transformation applied to the input signal. Most modern projection technologies, e.g., LCD, DLP, and LCoS, also utilize digital raster array devices to modulate light, so no curve tracing is possible. Finally, many modern immersive displays use multiple projectors to display a single scene, with overlap zones where the images projected by adjacent projectors are blended together to create a single, seamless image. In these multi-display configurations, all the projected images must be very precisely aligned in order to prevent the appearance of discontinuities.
When the projector optics and scan pattern are fixed, the image signal provided to the projectors must compensate for photometric and geometric distortion inherent in the display geometry, as is common with keystone adjustment. The process of producing an image using a non-planar projection is known as Nonlinear Image Mapping (NLIM). It is theoretically possible to use a 3D-2D projection other than simple linear perspective projection in order to compensate for display distortion, but in practice, scenes are generally modeled using polygons and rasterized using linear edge equations. Because a single polygon can take up a very large area on screen (if the eyepoint draws close to the polygon, for example), no amount of per-vertex geometric distortion can bend the edges of the triangle.
In the absence of parametric curve rasterization, the most practical method of distortion correction is to render the 3D scene using linear perspective projection and deforming the resulting raster image to produce a second, distorted raster image that is then displayed. This deformation may be performed in the projector (as with keystone correction); in a device placed in-line between the video source and the projector, which can be an additional video processing board in a larger system as implemented on the image generation systems manufactured in the 1980s by General Electric and Evans & Sutherland; a standalone video capture and processing system such as the one disclosed by Van Belle et al; or as a postprocess in the 3D renderer itself, which is the method of choice of most current implementations outside of the projector. The latter method was first disclosed by Arnaud, Jones, and Castellar, see U.S. Pat. No. 6,249,289, and further refinements of it were presented by Johnson to extend it to programmable shading primitives and by Bhaskner to extend it to high order surface rendering from its original piecewise linear implementation.
Regardless of where the deformation occurs, the process necessarily involves resampling the input image. Each pixel of the output image may cover zero or more input pixels, and due to the potentially irregular nature of the distortion parameters, the coverage footprint (the projection of pixels areas of the second image onto the first image) may vary considerably across the image. Some areas of the first image may be undersampled (less than one input pixel per output pixel) while others are oversampled (more than one input pixel per output pixel). Undersampling, which causes input pixels to be magnified, will exaggerate aliasing that is inherent in the 3D rasterization process. Oversampling, which causes input pixels to be minified, will introduce its own aliasing artifacts. Aliasing is a well-known phenomenon where objects will appear to scintillate, sparkle, or crawl as they move through subpixel distances across the raster grid.